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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments filed 01/14/2008 have been fully considered but they are 
not persuasive. 

Applicant argues that the amended claims limitation search engine is not covered 
by Boncyk. The examiner disagrees and point to the fact that the content server 
contains a search engine that searches for information on the image as claimed. 

Claim Rejections - 35 USC §112 

2. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 11-16, 25, and 28 are rejected under 35 U.S.C. 112, second paragraph, 
as being indefinite for failing to particularly point out and distinctly claim the subject 
matter which applicant regards as the invention. 

4. Claims 11 and 25 recite the limitation "the storage medium". There is insufficient 
antecedent basis for this limitation in the claims. 

Claims 12-16 depend on claim 1 1 and therefore are rejected. 

5. Claim 28 recites the limitation "the manufactured product". There is insufficient 
antecedent basis for this limitation in the claim. 
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Claim Rejections - 35 USC § 102 

6. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

7. Claims 1-8, 11-15, 17-19, and 21-28 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Boncyk et al. (WO 03/041000). 

Regarding claim 1, Boncyk et al. discloses: 

A system for image-based information retrieval from search 
engines, characterized by a) a terminal with a built-in camera that is 
connected to a remote data transmission network (8:22-10:5); b) a server 
computer on which an object recognition program is running, which 
analyses images sent to it and provides them with a symbolic indexing 
(37:30-38:1 and 18:6-10); c) a search engine that uses the indexed image 
to find information about the image and sends it back to the terminal 
(37:30-38:1 and 18:6-10). 

Regarding claim 2, see the rejection of claim 1 and note that Boncyk et al. further 
discloses: 

The system as described under 1) that is designed for mobile 
telephones or portable computers that have a built-in camera (6:7-19). 
Regarding claim 3, see the rejection of claim 1 and note that Boncyk et al. further 
discloses: 
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A city or museum guide that uses the system described under 2) to 
provide a user with information about objects of which he or she has 
previously taken a picture (2:5-7). 
Regarding claim 4, see the rejection of claim 3 and note that Boncyk et al. further 
discloses: 

The system as described under 3) in which positioning information 
is also used to appropriately limit the image recognition system (27:36- 
28:14). 

Regarding claim 5, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) that provides product information 
about products that have been previously photographed with the mobile 
camera (27:36-28:14). 
Regarding claim 6, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) in which the image recognition 
system is also able to recognize text characters or symbols (27:36-28:14). 
Regarding claim 7, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) in which the system is, in 
particular, able to recognize faces (27:7-9). 
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Regarding claim 8, see the rejection of claim 2 and note that Boncyk et al. further 
discloses: 

The system as described under 2) that is used to provide the user 
with additional information about advertising billboards (39:18-24). 
Regarding claim 11, Boncyk et al. discloses: 

A computer implemented system for image-based searching, 
comprising: a computer server, communicatively coupled with a network, 
that receives an input image from a user device communicatively coupled 
with the network (8:22-10:5); an image recognition system executed by 
the computer server and adapted to: determine a plurality of graphical 
attributes of the input image (37:30-38:1 and 18:6-10); match the input 
image to a reference image from a plurality of reference images stored in 
the storage medium (1 5:1-1 5:35), based on the plurality of graphical 
attributes of the input image and the reference images, each of the 
reference images having an associated symbolic identifier (37:30-38:1); 
and associate a symbolic identifier to the input image based on the 
symbolic identifier associated with the matching reference image (37:30- 
38:1); a search engine executed by the computer server and adapted to 
receive a query and to retrieve a set of search results associated with the 
query (9:40-10:5, URL); and a query processing system executed by the 
computer server and adapted to: receive the symbolic identifier of the 
input image from the image recognition system (9:40-10:5, URL); provide 
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the symbolic identifier to the search engine as a query, and to receive a 
set of search results associated with the symbolic identifier (URL lookup 
9:40-10:5); and transmit, via the network, a plurality of the search results 
to the user device (9:40-1 0:5 and 25:1-40). 
Regarding claim 12, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

User device comprises a mobile telephone having an integrated 
camera (6:7-19). 

Regarding claim 13, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

Server receives a geographic location of the user device in 
association with the input image (27:36-28:14); and the image recognition 
system is further adapted to match the input image to a reference image 
from the plurality of reference images based on the geographic location of 
the user device (27:36-28:14). 
Regarding claim 14, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a character recognition 
system (27:36-28:14). 
Regarding claim 15, see the rejection of claim 1 1 and note that Boncyk et al. 
further discloses: 
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Image recognition system further includes a facial recognition 
system (27:7-9). 
Regarding claim 17, Boncyk et al. discloses: 

A computer implemented method for image-based searching, 
comprising: receiving at a computer server, an input image from a user 
device remotely located from the server (8:22-10:5); providing from the 
computer server the input image to an image recognition system (37:30- 
38:1 and 1 8:6-1 0); receiving at the computer server from the image 
recognition system a symbolic identifier associated with the input image 
(9:40-10:5, URL); providing from the computer server the symbolic 
identifier to a search engine as a query (9:40-10:5, URL); receiving at the 
computer server from the search engine a set of search results associated 
with the symbolic identifier (URL lookup 9:40-10:5); and transmitting from 
the computer server a plurality of the search results to the user device 
(URL lookup 9:40-10:5). 
Regarding claim 18, Boncyk et al. discloses: 

A computer implemented method for image-based searching at a 
computer server, the method comprising: receiving an input image from a 
user device remotely located from the server (8:22-10:5); determining a 
plurality of graphical attributes represented in the input image (37:30-38:1 
and 18:6-10); matching the input image to a reference image from a 
plurality of reference images stored in a storage medium, based on the 
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plurality of graphical attributes of the input image and the reference 
images (15:1-15:35), each of the reference images having an associated 
symbolic identifier (37:30-38:1); and associating a symbolic identifier to 
the input image based on the symbolic identifier associated with the 
matching reference image (37:30-38:1); processing the symbolic identifier 
as search query to retrieve, from a search engine, a set of search results 
associated with the symbolic identifier (URL lookup 9:40-10:5); and 
transmitting a plurality of the search results to the user device (URL 
lookup 9:40-10:5). 

Regarding claim 19, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

User device comprises a mobile telephone having an integrated 
camera (6:7-19). 

Regarding claim 21, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Receiving a geographic location of the user device through the 
network (27:36-28:14). 
Regarding claim 22, see the rejection of claim 21 and note that Boncyk et al. 
further discloses: 

Image recognition system is further adapted to match the input 
image to a reference image from the plurality of reference images based 
on the geographic location of the user device (27:36-28:14). 
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Regarding claim 23, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a character recognition 
system (27:36-28:14). 
Regarding claim 24, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system further includes a facial recognition 
system (27:7-9). 

Regarding claim 25, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Image recognition system is further adapted to enable transmission 
of reference images, for use by the image recognition system, to the 
storage medium (8:22-10:5). 
Regarding claim 26, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Selecting a matching reference image from a plurality of reference 
images stored in a storage medium comprises: determining the graphical 
attributes in the input image represented by a plurality of trained attribute 
detectors (9:32-10:2); aggregating a plurality of confidence values 
received from the plurality of trained attribute detectors (9:32-10:2 and 
17:5-18:10); and determining the matching reference image where the 
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aggregated plurality of confidence values exceed a predetermined 
threshold value (best match, 9:32-10:2). 
Regarding claim 27, see the rejection of claim 18 and note that Boncyk et al. 
further discloses: 

Search results comprise links to websites (9:32-10:5), contact 
information (36:18-21), product information (40:24-27), translations of 
recognized characters (27:36-28:14), and other information related to the 
input image (39:18-24). 
Regarding claim 28, Boncyk et al. discloses: 

A computer implemented method for image-based searching of 
product information (40:24-27), comprising: receiving an input image from 
a user device remotely located from the computer server (8:22-10:5); 
processing the input image of the manufactured product on an image 
recognition system to obtain a symbolic identifier identifying the 
manufactured product in the input image (symbols, 9:12-31), the symbolic 
identifier comprising at least one of a product name or a product 
identification number, or a product code (9:12-31); providing the symbolic 
identifier associated with the input image to the search engine as a query 
(decode symbol 9:12-31); receiving a set of search results associated with 
the symbolic identifier, the search results including at least one document 
descriptive of the manufactured product in the input image (numerals or 
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text information, 9:12-31, 40:24-27); and transmitting via the network, a 
plurality of the search results to the user device (37:30-38:1 ). 



Claim Rejections - 35 USC § 103 

8. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

9. Claims 9, 10, 16, and 20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Boncyk et al. 

Regarding claim 9, see the rejection of claims 1 and 2 and note that Boncyk et al. 
further discloses in 40:24-27 that an image can be taken and used to retrieve 
maintenance instructions or repair history of a part of an aircraft. Official Notice is taken 
that it was well known at the time the invention was made to organize maintenance 
instructions for a part of a device into the form of a handbook, which would contain 
further maintenance instructions for other parts of the device. An advantage to doing so 
is that instructions for the device are organized for a person who desires to do further 
maintenance on different parts of the same device. For this reason it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to have 
Boncyk include organizing maintenance instructions for a part of a device into the form 
of a handbook. 

It is noted by the examiner that because the applicant has failed to timely 
traverse the old and well-known statement above, it is now taken as admitted prior art. 
See MPEP 2144.03(c). 
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Regarding claim 10, see the rejection of claim 2 and note that Boncyk et al. is 
silent with regards to allowing providers of information to make new entries to the 
system so that their data can be retrieved. Official Notice is taken that it was well 
known at the time the invention was made to update databases with new information. 
An advantage to doing so is that the person deciding to access the database will have 
the most up to date information available. For this reason it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to have Boncyk 
include allowing providers of information to make new entries to the system so that their 
data can be retrieved. 

It is noted by the examiner that because the applicant has failed to timely 
traverse the old and well-known statement above, it is now taken as admitted prior art. 
See MPEP 2144.03(c). 

Regarding claim 16, see the rejection of claim 1 1 and note that Boncyk et al. is 
silent with regards to updating the images and allowing future searching to include 
searching the updated images. Official Notice is taken that it was well known in the art 
at the time the invention was made to update image databases with new images that 
can be searched. An advantage to doing so is that the person deciding to access the 
database will have the most up to date information available. For this reason it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to have Boncyk include allowing providers of information to make new entries to the 
system so that their data can be retrieved. 
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Regarding claim 20, see the rejection of claim 18 and note that Boncyk et al. is 
silent with regards to updating the images and allowing future searching to include 
searching the updated images. Official Notice is taken that it was well known in the art 
at the time the invention was made to update image databases with new images that 
can be searched. An advantage to doing so is that the person deciding to access the 
database will have the most up to date information available. For this reason it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to have Boncyk include allowing providers of information to make new entries to the 
system so that their data can be retrieved. 

1 0. Claims 29-31 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Boncyk et al. in view of Waibel (U.S. Pub. No. 2003/0164819). 

Regarding claim 29, see the rejection of claim 28 and note that Boncyk et al. is 
silent with regards to using the method for identification of buildings name and 
information about buildings. Waibel discloses this in 1J0057 and 1J0061-0062. This is 
advantageous in that the user can annotate the images with the name and/or location 
for storage in a photo album. For this reason it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have Boncyk et al. include 
using the method for identification of buildings name and information about buildings. 

Regarding claim 30, Boncyk discloses: 

A computer implemented method for image-based language 
recognition, comprising: receiving an input image from a user device 
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remotely located from the computer server (8:22-10:5); processing the 
input image on a character recognition system, executed by the computer 
server to obtain text data indicative of the text (27:36-28:14); and 
transmitting, via the network, the text to the user device (37:30-38:1). 
Boncyk et al. is silent with regards to inputting the text in one language and 
outputting a translation of the text in a second user language. Waibel discloses this in 
110028. This is advantageous in that a user can identify a building or read a sign that 
the user is interested in knowing information about. For this reason it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to have 
Boncyk et al. include inputting the text in one language and outputting a translation of 
the text in a second user language. 

Regarding claim 31, see the rejection of claims 29, 11, and 15 and note that 
Boncyk et al. is silent with regards to providing the name and information about the 
name. Waibel discloses this in ^0058. This is advantageous in that the person's 
favorite foods, likes, and dislikes etc. can be ascertained. For this reason it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
have Boncyk et al. include providing the name and information about the name. 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to NICHOLAS G. GILES whose telephone number is 
(571)272-2824. The examiner can normally be reached on Monday through Friday from 
7:30am to 4:00pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Lin Ye can be reached on (571) 272-7372. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Lin Ye/ 

Supervisory Patent Examiner, Art Unit 2622 



